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ABSTRACT * 

This paper reports on an investigation of the" 
evaluation of teaching and. research at Stanford University, in the 
■School of Humanities and Sciences. It was found that professional 
colleagues had far more influence. on organi;sational rewards and 
penalties than did administrative superiors. Evaluations of research 
had far more influence than did. evaluations of teaching. Sii&e the 
faculty expended considerable effort *on teaching, "^m^ny wanted a 
better balance between effort and reward. For teaching, students were 
influential evaluators, since teaching performances^ were more visible 
to them, Foif ifeseafch, colleagues in other institutions .were 
influential evajluators, since the vii^ibility of /published research 
_made-«ie-boundari€S of the University permeable., Ovi^a 11, for both- 
teaching ane research, department colleagues and department heads 
were the moat influential evaluators. Faculty '.members « satisfaction 
with the evaluation proce.«?s was positively ass'bciated with the 
perceived dep^dence of their discipline on tHeory. Qreatef' 
deperiderice on theory in a discipline was associated with more g 
perceived agreement among evaluators, and more \agre^ent among 
evaluators was positively related to satisfaction with the -evaluation 
system. . (Author/HS) ' * \ ' ' . . - 
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^ Introductory Statement 

\ . ^ .. .. 

The Center mission is to improve teaching in Americai^ schools. 
Too many teachers stili employ a didactic style aimed at filling pas- 
sive students \i[ith facts. The teacher's environment of ten ^preyents * 
him from ^changing his style, and may ind'e§d drive him out of the pro- 
fession. And the children of the poor typically suffer from the worst 
teaching. * ; * 

The Center uses the resources of the behavioral sciences in- pur- 
suing its objectives. Drawing primarily upon psycholpgy and sociology, 
but also upon other behavioral science disciplines, the Center has' 
'formulated progr^s.of research, ^development, demonstration, and dis- 
semination in, three' areas. Program 1, Teaching Effectiveness, is now 
developing a Model Teacher Training System that can be used to train . 
both' beginning -and 'experienced teachers in effective, teaching skills. * 
Program 2, The Environment for Teaching, is developing models of school 
organization- and ways of evaluating teachers that will -encourag^'e teachers 
to become mor^ professional and more coinmitted. Program 3,. Teaching 
Students from Low-Income Areas, is -development materials and procfedres 
for motivs.ting'both students and teachers in low-income schools. 

t 

^ Tha research reported here was • conducted in association with the 
studies of evaluation and authority within the Environment *f or" Teaching 
program. , ^ - 
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Abstract 



This paper reports an > investigation of the .evaluation of t,eaching 
and research^ at Stanford University in the School of Humanities and 
Sciences* All the data were collected in 1968 ftom interviews of a 
• sample of 100 faculty, stratified by rank* ^ ^' ' * - 

It was found that in this professional bureaucracy, prdfessional 
colleagues had* far more influenc^on organizational rewards and pena;Lr- 
ties than did administrative superiors • Evaluations of researcl^-by 
all evaluators combined — had far more influence than did evaluations 
*of teaching • Since the faculty expended considerable effort on teaching, 
many wanted, a better balance between effort and r^wrard* For teachings 
students- were influential* evaluators, since teaching performances were 
most visible to them* Information volunteered by the faculty '^indicated 
that they were divided 4bout the quality of student evaluations. For", 
research, colleagues in other institutions were^irEIuential evaluators, 
since the visibility ^of published research ^ade tjie boundaries of the 
University permeable, , Younger faculty/ more mobile, .emphasised' res.earch 
more than older faculty did, OveralL-, for both teaching an4 research, ^ 
department colleagues, including department heads, were the most influen- 
tial evaJLuators, • ' / ' ^ 

V 

Faculty members' satisfaction with the evaluation\process was posi- 
tively associates; with the perceived dependence of their -discipline on 
theory; Greater dependence on theory in ^'discipline was associated 
with more perceived agreement among evaluators^ and more agreement among* 
evaluators v^as •positively related to satisfaction .with the evaluation 
system. , This finding appears to be the basis, for ^the -professional em- 
phasis, on collegial evaluations, that is, evaluations" based on a shared 
body of knowledge, 

^ . ' « .\ ' 

Tlie study reported here provides the baseline for a current study of 
the effecttof attempts made since 1968 to increase the^weight given to ,^ 
teaching at Stanford, ^ • . ' * 
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THE .^VALUATION OF UNIVERSITY TEACHERS : ■ AN APPLICATION 
OF A -THEORY OF EVALUATION AND AUTHORITY * 



Robert R. Hind, ^Sanfori M. Dornbusch, and*W. Richard Scott 



A theory of evaluation ^nd autfiority first put forth in 1967 seeks 

to. explan the stability and effectiveness of authority systems in or- 

' 1 . * ^ \ ' * 

'ganiz,atior\s. ** Since then we have applied the the'bry to 'Roman Catholic 

priests, "intern^ and residents in hospitals, .teachers in public' school s^ 

and alternative (free) schools, varsity football players, a physics re-,. ' 

, search group, an' electronics assembly line, the st'aff of i student news- . 

paper, and workers in a\ factory and a hospital in Nigeria. Thege emp'iri- 

cal 'studies tested^^ portions oL the* theory and led to its reformulation. 

' Tfhis paper attempts to show how some aspects of the theory apply to the 

' ' 3 

process by which r university faculty is evaluated. Our .empirical find- 
ings also have ii^Xications for directiLng change in a university. 

Every organization is a *power structure oriented toward controlling 
and'^^coordinating activities. The structure may be hierarchical, with 
control exercised by bureaucratic superiors, or collegiai-, with cohtrol 
exercised by fellow professionals of approximately . equal rank. Our , - 
studies found ^considerable variation in control structures, even among 

professional bureaucracies"* For Roman Catholic priests, the hierarchy 
■ » <» • 

Robert R. Hind^^Director,* Western Region, of the Academy for Educa- 
tional development, Talo Alto, Sanford tj. JDornbusch is Professor of 
Sociology at Stanford and a Rese^ch and Developr^ent Associate of the"^Cen- 
^ ter. W. Richard Scott is- Chairmen of the Department of Sociology at Stan- 
ford and a Research and Developmei^t: Associate of the Center. 

1 • * . ' ' 
W. Richard Scott j Sanford M. Dorijbusch,' Bruce C. Busching, and James 

D. Laing, "Organizational Evaluation and Authority , . Admind^strat j.ve 

Science Quarterly , Vol. 12 (June 1967), 93-1-17. " ~' \ ~ 

V / 

2 ' ' • ' 
^ Sanford M. Dornbuscb .and^ W. Riphard Scott; Evaluation and Authority 

(New YorkV McGraw-Hill, forthcoming). , 

- ^ Robert R. Hind, ''"Evaluation and Authority in a University^ Faculty" 
(unpublished Ph.D. dissertation,* Stanford University, 1^68). 
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assumes more importance as '.priests get: older. Among public school teach- 
ers, hierarchical conttols are weak and collegial controls are almost 
nonexistent • This paper presents evi-dence that at Stanford University — 
a professional "bxireaucracy—collegial ''control is much stronger than 
Bureaucratic control. . 



Research Procedures 



'^3 Sample 



1 



In 1968 ^e^nterviewed a random sample (stratified by rank) ^ of 100 ' 
faculty in £he School of Humanities and Sciences at* Stanford University,* 
A school of this sort, often called the college of.arts^and ^sciences, is 
the core of most major American^universities. The ^ields^represented by 
these faculty members were similar to those in the traditional liberal 
^rts. coliegejN Since "our data were^ gathered, attempts havfe been made at 
Stanford to increase the emphasis on teaching in the'evala^S^xon process,^ 
Thus, the results given in this paper may not depict the CuriWfe' situation 
at Stanford. ^ . * x 

In afiy case, this paper is not meant to be descriptive.. We hope our 
results are abstract enough to be applicable to ia wide variety of academic 
institutions.. Our propositions and hypotheses are intended to be geneij- 
alizaBle. Therefore, it is. appropriate to state here some of Stanford's 
characteristics so that the skeptical reader can have a basis for limiting 
the applicability of our findings. 

Stanford University is an independent, nondenominational university 
in ^northern California subutb. It bas 12,000 students, about half' under- 
graduate and half graduate students. Admission is. highly competi^tive. 

About half of - the 1,000 r'eg^lar faculty are in the School of .Humanities* 

* • / * ' 

and Sciences from wliich we drew our sample; the.r^st are in/six graduate 

and professional schools. These figures ^ere very nearly t^e same in 1968. , 



A follow-up study of their relative success is under way. 



Stanford was recognized as a good regional university from its found- 
ing in .1885,. until the late 1940^s. An agfer^ssive ceivbral administration 
"then combined government research grants, gifts from private donors, and 
active recruitment of faculty to construct one .of the :r(ation*s leading 
universit;Les, with^a strong research and graduate training program. oAs 
a result 06 this recruitment and expansion, most of t;he faculty in^our 
sample had been at Stanford less than 10 years, although a sizablq minor- 
ity, 15 percent of our sample, had been at Stanford .over 20^ years.? Younger 
men- selected primarily 'for their research dominated the faculty* /ihe or- 
ganization of the faculty*was, and still j^s^ strongly departmental, with 
'(each department controlling its own instructional program. The central . 
administrations of the University and pf the School of Humanities and 
Sciences exercised their authority primarily through the allocation of 
resources to the departments. . , ^ , ^ 

An analysis of the educational process at Stanford helps to illustrate 
the decentralization of power at research-oriented universities, £\i^ per- 
meability of universities — social systems whose sets of influential eva- 
luators may be far beyond the individual campus* — and* 'the difficulties of 
redressijig the imbalance between the emphasis on research and^the emphasis 
on teaching. Clearly, • Stanford is close to one ^treme of university^ 
organization, in the United States, yet it is representative of some major 
institution^. It can be used as a baseline for c^omparison with studies 
of other universities, colleges, and* junior college^ 

The Interview 



5 



Each professor in the sample was asked a series of questions ^bpi^t 
the influence/ of the evaluation of four tasks in determining university 
rewards, and the ident-i^ty and influence of various ^valuators. Since Stan 
ford has an annual review of all salaries, with-4io fixed scale for any, rank, 
faculty members readily ^recognized that evaluations directly and indirectly 
affected the reward system. By contrast, the public elementary and/ 

— 5 — - ■■■ ' ■■ ^ /■ 

Charles E. Bidwell, "The School as a Formal Organizations/in James 

G. .'March, ed., Handbook of Organizations (Chicago: Rand' McNaLly, 
1965); Burton R. giark,> The Open Door CoTlege (New York: McGraw-Hill^ ' 
1960X; Talcott Parsons and Gferald M- Piatt, "The American Ac/demic Pro- 
fession^ (multilithed, 1968). 



secondary school teachers whom we fi'tudied had difficulty conceiving o^- 
a system of rewar ds tha t had minimal variability and that depended on 
infrequent ^^lu^tions base^ on non-oommunifeated criteria. Faculty mem- 
bers at. Stanford knew that^ they 'were being evalu&f-ed and that evaluations 

. mattered. ^ . ^ ^ - 

The questions*, w^ere designed to, elicit informatio\i^about the individual 

.exp^iences and perceptions" of respondents, not what they thought about 

institutional processes in generalp The list ^ of questions was repeated 
' * ^ > * * *' 

for each of the princ4.pal tasks to which respondents. attacHed-^ inportance. 

Where strong disparities, or dissatisfactions were iioted, we probed, for 
further*'details,, noting these along wi^i volunteered- c'ozmnegrfcsl ^ / 

All interviews were -concreted* by the senior author^f - this paper, 
who was then Staff Director 9fN>he Study of Educationist Stanford-. " TheS;, 
.research reported here* was, part of a program' self-study. Oijly five 
. faculty members, tour full professors and one associate professoij", chose 
not ^o participate in fthe study, ,aud they we^^r^placed by alt^nates 
also, selected randomly. The-noncoopeiating professors were not so^numer-^^ 
ous as to distort our results. ^The 34 full prof essors^"^ associate 

professors, and 33 assistant professors were representative of -their 

> „ " , -. ^ ' 

iganks lii the faculty of^* Humanities and Sciences. 

Because the interviewer hel<i an administrative positioti ^,the Uni- 
versity, assurances "of complete anonymity and confidentiality were con- 
\ veye(J, ^n the letter inviting participation and in the oral introduct^jLoii 
to the interview. We be^eve that these assuij&nces were' accepted, -and 
our impression is supported by the large number o*f responses that criti- 
cized adniiniiitrators and administrative procedure! 



Applying the Theory , 



The University ag a Professional Bureaucracy 

In our theory, prganizations are, defined as power structures in which 

Some participants are given differential influence, over organizational 

* /" * • ' 

rewards and penalties in order to control other participants. We believe 

that evaluatiQns are necfessary, but not suf f icient,>v f or* the control 



of behavior. distinguish between the^subjectjfve importance of an * 

evaluator or^ e.valuation' and the 'influence of the evaluator on organiza- . * 

tional rewards! and- penalties. ^ , 

If participants care about organizational rewards ^atid penalties, 

then eyaluators "wha have more influence . over theit distribution ^/ill J^te 

more important to participants than^iJill evaluators who have less in- - ^ 

fl\ience, » Similarl/y, evaluations Tnat-.have more influence over the dis- 

) ' * * / 

I tribution of otganizational rewards and penalties 'will be more /important. 

i < ' - 1 % 

to participants than will those that have less. Accordingly, to- the exr- 



teht .that participants value organizational rewards, influential evalua- 
tors and 4^aluataons will be perceived as, important by the people being 
§valu^tea;. Not ali ipaportant evaluators need be influential, however/^ 



Our i^arenis, .our spouses, and our former teachers may be important to us 

even though thfey have no effect on'v^at happens to us within the .organl- 
. • ' ' ' • ' • O ' ♦ 

ication. • * . * . ' 

* o * . 

4 Therefore, given our p-royious research ^showing that influential 

evaluators become Important and that Unfluential evaluations are deemed 
* . * . ♦ « • 

J-mportapt, we concentrated in this study on the extei^t to which, evalua- 
tors and evaluations influence the distribution of organizational rewards 
and penalties. The bases *for influence may be varied and the means of 
exerting influence may shift, but we ar^e simply studying the end product: 
the impact on the distribution of organizational rewards and penalties.. 
By this siniplification considerable detail is'^lost, but comparability 
across studies and. methodplogical simplicity are gained. 

Data were collected on the evaluations of **four^^acu*Lty tastes: 
te^chipg, research (or scholarship) ^ university service, and 'external 
service. The last two tasks 'were not seen by the professors as having 
much influence upon the distribution. of university, r^ewards and penalties. 
The evaluations of university service were perceived to be* "very influential" 
"or "extremely influential" by only eight percent of the sfaraple, 'and ex- 
ternal service by* only three percent. .When asked how much influence , 
evaluations 5r ithese ;two tasks' should have,* the faculty showed no desire 
to raise their influence. We can, therefore*, safely l:onc*entrate' oil the 
^evaluation of only tt^ro tasks: teaching and research (with r^^search 



including what some faculty call scholarship) . These lare they)rincipal 

J- ■ 



tasks of the Stanford faculty. Research was considered "very influential" ' 
or "extifemely influential" in the reward system by 78 percent of tlje -fac- 
ulty, teaching by 20 percent. The- great emphasis^ on research,' of course, 
provides a c^ontexf'for comparison with other institutions. 

J Professionals are oriented toward control^by standards .shared bV 
members, of t^^j^r own- occupational group, ihey attach greater importance 
to evaluations by their professional colleagues- than to evaluations by non- 
professional?. Stanford' University is" a professional bureaucracy, and its 
structure of influence "emphasize's cbllegial evaluations. We hVppthesi'afed ' 
th|t professional colleagues would be the most influential evalH^r^ors at 
Stanford-;-more influential ^than s\iperiors^ in .phe administrative hierarchy. 
Each professor was asked' to/\ame 'those .persons' who, for each task, 

*made evaluations that might influence University rewards. Each professor 

\ ^ ^ ' \ ' I 

^was shown the following list from wiiich to .select influential evaluators, 

^1. Students^ . \" ... | 

2* UDepartment head ' \ . '. 

3, Department colleagues' . ^ " i 

^ \ 4, Facility members in oth^irdepartments J 
'\. 5. Members t>f the jsame 'discipline in' other institutions 

6. • Persons having a say«-in government or foundation jgrants 
7; Other outsiders (please nampX • , 

8, , bean of the p^chbol and"*his staff - . j 

9* AppoiAtmijeht^and Promotion Conniiittee i ' 

10; Other f^aculty cpnunittees (pleaisf* name) I " 

11, Provost'^and his, staff ' * ^ ^ 7 | *^ 

12, President * ^ . . ,L ' 1 

13, Trustees ; ' ' '"^^ 

We th^ri asked' each professor how. much each of his ^valuators influenced 
the distrit^utlon of - organizational rewards and penalties. Table 1 shows 
the perceived influence of evaluators fbr the- two principal tasks, witli i 
some groupings of /evaluators who are.-3f^ss frequently considered influen- ' 
tial. Table 1 combines two kinds of ihflilence\3 the influence of evalua- 
tions of each task and the comparative' influence .of groups of evaluators. 
We have already noted 'that , evaluations of research are. far' more likely 
'to be "very, influential". or "extremely influentil"' than are evaluations ' 
of, teaching. Therefore, no group of evaluators is likely tp be high in 
influence when they are evaluating tha task of te^^ching, Xf we combine 
the percentages labelled as "slightly irifuential,A "not at all influentlalT" 
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Perceived Influence of Evaluators 
(N = 100) . . ■ 
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' — ^y^xtremely Very - 



• Moderately Slightly \ 
Influent.-' al Influential -^Influential Influential* 



NQt^t -All Not 
Influential Namei 



Teaching 
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1.. .Students 
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2. Department .'head 
3\ Dept.. colleagues" 
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4; Other faculty • 
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or "not named" at all, we have the extent to which each group of evalua-, 
to^ was considered low in ijifluence. For teaching as a task, the 
majority 
research. 



of the •faculty considered every .evalyator low in "infliience. For 
on ^tlie other hand, there weye* three groups of evaluators who 



^were not considered low in influence by; the faculty. Therefore, when 
we consider the perceived infl^uenceopf evaluators, we ar^ looking paly 
at the relafciye influence'^of^ eva0ators of'.eaqh taskt (.Evaluators. of 
research, irt genetal, hkve more influence; yet we^can Ue^am much from 
/comparing groups of evaluators with respect to the influence of their 
evaluations of teaching as ^well. * , 

The ^ata support the hypothesis that professional .colleagues- are 
more influentl^ than administrative superiors, fabl^-^l shows that many 

^more professors nam^ their departiien^ -.colleagues 'as influential evaluators 
than name any administrator in the School or^University^T"^\;Administ)t 
other than the departmei>t head are tiot oniy^named less often,, but they 

^are also i>erceived as lower iV influence thatf cplleagiies. For 

example, oiLLyfoi^n percent of the faculty* saw the Dean^ and his* staff as 

. "yejry i|{flu6ntian or "extremely influential" evaluatprs for the task of " 
t5^^chiTig,.-,and onlyo22 percent saw the Dean add his staffs bs ^Very in- - 

Ipentlal^r qt "^-tremely* influential" in the evaluation of research. 
Tl](ie department heads, department colleagues, and students wer.^ all per- 
cfeived as having more i;nfluence-".than the "Dean for the task of teachif 

J^or research, the department .heads, department colleagues, and colleagues^ 
at. other institutions wefe all\rated higher in influence by the faculty 
than was the Dean. This lower Influence pf administrators is probably 
related to the combination of two perceptions: that administrators do ^ 
not know what he appropriate evalua'tive^riteria are; and that faculty 
performanc€!s are not visible to them. " * 



Table 1 also shows "that the department headT^who 



is both colleague 



and administrative superior, has a critical role.- His "prof essional 
credentials serve to render hi^^valuations acceptabl^ tp hiT^^colleagues, 
;whij.e^his bureaucratic office requires him to gather certain -ki^ds of 
fin'f btmatibn about the work of his colleagues , attempt to exercise certain 
kinds of controls over them, and help to decide how rewards and penalties 



f 

I 

V 

• . ' -9- . • 

are to be .allocated ainpng them. ' It is the creation of-'special rples such 
as departmeht head that leads us to conclude with Etzioni that "a large 
amount of control over professional performances has been transferred 
from the professional community to the professional organization."^ 

.<Since the faculty* s-emphasis 'on evaluations by professional colleagues 
is central ta this paper, we examined the responses of faculty in the 
different ranks to see p.f^,t1ier^ was any major variation in th^ percep- 
tions of assistant pro^ess9rs, associate pnpfessors, and full professors. 
Assistant professors .are rewarded by tenure, promotion, and salary in- 
creases; associate prof essors -'are given promotions 'and salary increases; 
and full professors receive only salary increases. These differences in - 
available sanctions had no impact on our findings. The pattern »of influence 
among evaluators was^ identical for the three ranks. For each rank, de- 
partment heads, -department colleagues,- and students were all perceived 
as^iaore influential^ than ^the Deari for the evaluation of both teaching and 
research."^ So, -despite the differences in rewards and penalties, faculty 
members in each rank agreed on the relatively low Influence of even the 
.most. influential administrator. ^ ' [ 

Visibility and Fermeability ^ / *< 

Our theory' states that evaluations based on .greater visibirity of 
performances, and outcomes are more likely to^^be considered soundly based 
b^^ the participant.' Inj addition, participants will want evaluations to 
-be soundly based so. that tVexi; performances can affect the ev^riCua^ions. * 

We defined visibility as the frequency or .proportion of perfotmancea 
observed or outcomes reviewed by an evaluato/. This gave us four measures 

J • 7 

of visibility in our ^tudies-^of nurses and public school-teachers. Each • 
spe^€>lc medsufe of visibility^ of work to an evaluator produced- equiva- ^ 
lent results.. Among i^rses and teachers, each measure pf visibility was 




Amitai Etzioni ^ A Comparative Analysis of Complex Organizations. 
(New York: Free Press ^f Glencoe, 1961), p. 259. 

• . • ■ 

7 

Gweri Marram, Sanford M. Dornbusch, and W. Richard 5cott,- ."The 
Professionalism of Elementary School Teachers, Teaming, and the 
Visibility of Teaching" (tentative title; article submitted for 
publication). ^ ^ ^ > , 



strongly correlated with the perceived- soundness, of evaluations. The 
evalua>:ors to whom' work was mqre visible were considered to have more 
soundly based evaluations, and their evaluations were considered more 
important by the nurses^ and teachers. The only evaluators who were 
high in influence and loWer in importance than would bje expected were 
school superintendents and directors of nlirsing, to whom the work was . 
not visible. . . . • • , 

We did not directly .collect data on visibility from the Stanford 
faculty, but the impact of visibility is clear. Since^ Stanford. is a 
professional bureaucracy with the current evaluative structure close to 
4:he preferences of the faculty, visibility affects the current system 
of evaluation*;- Students, for example, were seen as -major evaluators 'of 
teaching. .Students were tKe .most often named ev^uator^ for teachfng, 
just behind the department head and tied with department colleagues in 
the reported influence of their evaluations; However, even this rela- 
tively 'large perception!' of student influence in the .evaluation of tea 
actually underestimates student participation-. in the evaluation process. 
In a university,, teaching is often visible only to students. Therefore, 
students are the^ primary source* upon which other evaluators base their 
judgments of ^teaching. When they' were asked' the sources of information 
u'Bed by department heads and department colleagues in the* evaluation of 
teaching, the faculty ^repotted that students were the source of informa- 
tion f or ^1 percent of the department heads and 93 percent of the de- 
partment colleagues. " . - 

Yet the high visibility of .teaching performances -to students con- 
flicts with the faculty \s emphasis* on evaluations by professional colleag 
Although visibility increases the likelihood bf sound evaluations by stu- 
dents, their ignorance of professional standards reduces faith in th^ir 
judgment, "faculty members- were divided on the* si^ject ^of students*as ' 
good evaluators o£ teaching. We asked na specific questions about *the 
quality/ of any of the evaluators ^ yet over -half of our sample commented 
on it with regard to students. ' Twenty-ei^ftir^ercent volunteered comments 
which indicated that students were good evaluators of teaching, while al- 
most an identical number, 29 percent, said that students were poor or 



.unreliable evaluators -of teaching. This finding ia.in sharp contrast to 
tliat of a related study of nurses, and/ teachers. Amon^g- .nurses and teachers, 
there is a very high correlation between the visibility of their ^rform- 
ance to an evaluator and the desired influence of that evaluator. But 
the "evaluators in these studies were grof essionally trained' nurses, head 
nurses, teachers, .and principals. 

We defined organizatiorfal evaluators as thos^ persons who influence ' 



the^diftfibution o£ organizational rewards -and penalties. Organizational 
evaluators need not be within the traditional boundaries .of the organiza- 
tion. Tfie greater the Visibility of task performances *or outcdmes to 
outsiders, the more likely it is that the jjoundaries of the organiza- 
tion will be permeable and that outside evaluators will influence the 
distributioiT of organizational rewards and penalties. Research is visibre 
to colleagues at other institutions through the publication of results, 
whereas teaching is not. ^ Seventy-percent of the faculty named colleagues 
at^her institutions as organizatibrial evaluators of research and 40 per- 
cent saw these distant colleagues as "very influential" or '^tremely ^ 
influential." .But since teaching' is not highly visible to outsiders, 
the organization is less permeable with respect to the evaluation of 
teaching. Only 15 percent of the faculty named any outsiders ^as organiza- 
tional evaluators of their teaching, and just four percent saw them as 
Vvery influential" or "extremely influential." If. the two sets of evaiua- 

ns are combined,' we find that 19 percent off all evaluators named 

T ' J ' / 

(for: both research and teaching) we're located outside the organization \ 

whqs^,^wards.and penalties were being distributed. Yet these 19 percent \( 

were actually influencing thfe allocation of those sanct;Lons. ' -The per- / 

meability of the Univei-;ity reported for research is striking because we / 

are considering only influence on the distribution of^ Stanford's rewards,/ ' 

not outside honors and awards. Outsiders* influence ra'^^e indirect, via 

cbmpetihg offers or reflected reputation within Stanford, but^it nevert^er 

less influences Stanford's rewafi^ system. . " ■ 

^ Younger faculty wer^ particularly likely -to emphasize those tasks-- 

research and scholarship--that would be most visible to their colleagues 

at other institutions. At Stanford, as at many colleges anfi universities. 



there had been much public discussion- about, the relative inattention to 
teaching among the faculty prior to our study. It was expected that 
this shift in values would be reflected in more concern ab6ut leaching 
among younger processors. On l:he contrary, younger faculty tended to 
be mor^ mobile and to care more about participating in a national and 
inte'rhatiimal job market. The visibility of research to other insti- 
- tutions was so powelcful a force that the increased, emphasis pn teaching 
was'nowh^re evident for younger faculty, if it existed at all'. We 
examined our data^to see if this result was spurious. . However*, within 
each rank, younger faculty emphasized research more^ than older faculty.. 
Younger professors appear to be emphasizing the task for which, the or- 
ganizational-boundaries are most permeable. 

The^permeability of the university to outsfHe evaluators is-'also 
illustrated by the influence ^f the awarders of .grants in t:he evaluation 
of research. The influence o^ grant makers is. not very high, with only 
eight percent x5f the faculty viewing them "extremely .influential" or 
"very Ihfluential" evaluators of research. Nevertheless, the perceived 
influence of grant makers did vary with the amount of money received 
by each department. For departm^etvts whi<Ih received less than $1^000 
per year per prof essot' in grants, 92 percent assigned no influence to - 
grant makers in the evaluation of research; and none saw them as "very" 

or "extremely" iiiflufent^al^ For departments receiving from* $1,000 to 
* ' * ^ ' " ^* 

$9> 999 'per year per professor, 80 percent saw grant makers .as without 

influence and only eight percent saw them as lllgh in influence. For ' 

departifients that received $lc/,000 or mbre per year per professor in 

research grantfs, the) proportion seeirig gisant makers as hot influential 

declined to 69 perdent, and the propoVtaon seeing them as "very" 

or "e:^tremely" influential rose to 17 percent. Thus, /although the 

proportion who perceijVed grant makers as -high in influence was never 

large, the amount of funds received by a department was directly related 

to the perceived influence of grant makers in that dei{artment. ^ 
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Dependence Upon'a Body of Theory . » ■ 

It was ♦hypothesizBd that satisfaction with the evaluation process 
would be related to perceived agreement ampng- evaluators; that is, if 
evalirators are seen as agreeing, ^ the professor is less likely to be dis- 
satisfied wi^t^h' their evaluations • To measure the extent of perceived / 
agreement we asked, ."How much .agreement 'do you think there^is ampng peo^ 
pie in your .discipline in their evaluations of (task) done by their * 
colleagues?'^ Somewhat surprisingly, agreement among evaluato^ as 
perceived by ^e faculty was approxlma.tely equal for *the tasks of re- 
search and teacliing, ^ There were only minor variations among fields, ex- 
cept, for the arts, which tended, to be low in. perceived agreement among 
evaluators 'for both teaching and research. There was also. a slight 
tendency for faculty in the natutal sciences. and. mathematics to per- 

ceive a higher level *bf agreement among evaluators for research. than 

♦ • * 

'for teaching. , ^ " 

Arj/analysis of the relationship between perceived agreement among 
colleagues* evaluations of research satisfaction with evl^uations 
by those colleagues reveals a gamma of .59 betweerf high agreement and 
high satisf act^.on. ' Eighty-three percent of those who perceived high eva- 
luator agreement among colleagues were high in satisfaction with evSaGa- 
tions, compared with 56 percent of those who perceived low agreement * 
among their colleagues. *^ » ^ ^ 

Our ^heory views satisfaction as partially .the product of perceived 
agreement^ ai^n§ evaluators, but clearly an interpiretation emphasizing 
cognitive consistency w^ould alsoifit the data. Such a' cognitive inter- 
pretation would portray satisfied* faculty as more likely to perceive 
* ... 

agreeme^. Our theory specifies a direction from agreement among eva- 
,luators to satisfaction among faculty, and the next set 'of findings ^ 
ndicates some basis for this view.. |^ 

Greenwood, among others, has noted that one of the criteria for a 

8 ' 

profession is its dependence on a body of theory. We therefore hypothe- 
sized i:hat, in a professional group, agreement among evaluators. would be 

\ < ' ' I . 

*' 8 \ ' ^ . 

E'rnest Greenwood, "Attributes of a Profession," Social Work ^ Vol. 2 

(July 1957), 44-55. . , ' 



more likely when the discipline wasr perceiA)'ed to be based upon\a body of 
theory-. ^he body of ^h«=j / would provide a common set of standards for 
evaluation. ^ - ' ' 

To get- a measure' o^ dependence upon- a body of theory, we asked, 
"Some academic fields or disciplines are seen as strongly based on a 
systematic body of theory which serves as the basis for research. hypo-\ 
theses and dominates teaching and scholarship. Other fields are less 
reliant on a central bodysof theory. To what extent does your own field, 
have a central body of thebry to givide research, scholarship, and teach-r 
ing?!' For the 77- respondents^who answered both questions, we obtained 
a gamma of .45 relating high d^eudence upon theory and high perceived , 
agreement among the evaluators of their research. Sixtjf-f-our percent 
of. faculty^^o saw'their discipline as "extremely depefident'Vor "very^ 
dependent" upon theory perceived high agreement among their evaluators^ 
Only 40 percent of ±^se who felt their discipline had a' low 'dependence 
upon theory perceived high agreement among their ^valuators. Dependence 
upon theo^ is more prevalent in the -natural and social ^sciences th^n in 
the humanities. ITone of the natural scientists, ^17 percent of the mathe- 
matical scientists, and 29 percent of the social scientists, saw their " 
field as "slightly dependent" or "not -at all dependent" upon theory,, 
compared to 51 percent of the faculty in th^ humanities'. ' . 

From this /finding, we predicted that overall satisfaction wit^ the 
evaluation process would be positively related. Co perceived dependence 
upon theorywithin a discipline. Since mani^'res^ondents^"" gave- middle 
responses, we trichotomized the responses according' to dependence upon 
thepry. .We- foupd that overall satisfaction -with evaluations for those 
professors in disciplines perceived to be highljr dependeiyt upon theory ^ 
was 60 percent; for tihose in disciplines with medium dependence upon 
theory it was 52 percent; and for those ,in disciplines witli^ low depen- ^ 
dence upon theory it was 38 percent; These data support the* view that 
satisfaction with the evaluation process -is relate^ to perceived agree- 
ment among evaluators,, *which^ is in turn^related tc^' .perceived depe^tidence 
•upon a body of theory. This finding appears to be'the basis for the 
professional emphasis 'upon collegial evaluations; that is, evaluations 
based upon a shared body of knowledge. %c " s 
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• Our studies of nurses and public school teachers provide supporting 
evidence for the relationship between .a shared body of knowled|e /and ' - 
collegial control, teachers, who .overwhelmingly consider "personality" 
m6re import^ than training in producing a good teacher^, are reluctant 
to'pertait evaluations by colleagues to be inflyential. Nurses, who con- • 
' sider their training vital, are oriented to the evai«a£ions of other nui>ses.' 
Thus, it appear;s^hat a shared body df knowledge isa^ a^^^r bjsis for t^e 
collegial control that is presumed central for professionals. 

Effort and .the Evaluation Process ^ - ' . 

. / . / ■ . ^ " ' ^ 

It ^as already been argi^ that if participants care about organdza-r 

'tional rewards and peijaltieesP^hen evaluations that- have more influence 
over the distribution* of organizational rewards and penalties will be 
more important to- participants than will those evaluations tha,t have less * 
influence. 'The data from our stud^of university faculty* partially support, 
and partially do not support, this proposition. The^ effect of-^he organiza- 
tion^^ reward structure upon faculty effdrt is pajrfly vitiated by the , 
. :^nternalization of professional Standa^s, which remain powerful, and In- 
dependent of the evaluation-reward systemi The Stanford faculty recognized 
that. evaluations of researdh were far more influential than evaluations 
o^ teaching. Seventy-eight percent- saw evaluations of 'research as. "very 
inkluential" or "extremely influential,"" compared to only 20 percent for 
teaching. Yet most of the .faculty reporte^ that ESffey spent more time on 
theWarlous ffSx^s of teaching ^than on tesea^ch. This ^indiug app^rs tc% 
defjr\ the attempts of the eV.aluation system tb^cijcitrol th^ allocation of 
effort. Thus^ the discrepancy does produce straiit>nd^a desire/ *f or better 
balance.! . / ' ^ 

Firs^, 40 percent of the faculty wanted to increase the amount of time 
devoted tc research, compared to only 16 perc^ who wanted to reduce their 
rese'arch-^uffort. A substantial majority of the faculty wanted to continue 
with Vhe Liount of^'time currently devoted to teaching, with' slightly more • 

— — 7^ • 

Marram, Dombusch, and Scott, o£. cit ; 
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. - ■ ^ • . . .: . I 

wiS|ilng to Reduce their teaching time /than wishing to increase it. These 
desired shifts in allocation of time /would slightly increase time for re- 
search and slightly reduce time for teaching, a preference that would, on- 
the average, produce a* somewhat bejiter balance between effort and reward. 

Second, the faculty wanted to increase the/influence of evaluations 
Qf teaching. Fifty-one. lierce^it ^oE the professors felt teaching should ^be 
"very influaitial" or jf" extremely influential," a marked increase. Yet 
an ^ven higher proportion, 67 percent, thought r^earcb should 'be "very" • 
\^ "extremely" influential. ^ Thus, although the percentage of faculty who' 
felt that teaching should b> highly influential is more than double the ' 
I number who now see it as high in- influence, th^' decline in the 4e^red 
♦ influence of research- *is not so, great* as to make teaching and research- 
equal in influence. The faculty viewed research as the most ^^fluential ^ 
task and believed , that it shoi4d l^ontinue to be so. The desire for bal-A 
Knee is indicated, however^ by"£ii^ strong desire to increase the influence 
of ^valuatioYis of teaching. Fif ty-rthfee percent of the faculty wanted to 
» increase the influence .of teaching upon rewards, and only two percent - / 
wanted to (decrease it. Faculty members were Peeking to }5alance their 



expenditures of effort with, the influence of their t^ks upon rewards. 
Teaching seems to .have been intei^nalized as part of a professional 

-conception of an apptopriate professional role. Despite this feeling of 
professional responsibility ^r t|^ching, there "was, as\predicted, a 
tendency among faculty members *?to spend more time on those* activities 
whidi were believed. to. benefit them mo3t. Among those who saw teaching 
as "moderately;" "very," oir "extremely" influential, ^65 percent were above 
the median on time spent teaching undergraduates and 77 percent wer.e above 
the median on time spent on graduate teaching. Foir those who saw teaching 
as "slightly influential" or "not at all influential," only 43 percent \ 
were above the median' on time spent on undergraduate teaching and 47 percent 
above the median on time spent, on graduate teaching. " . * ; 

The same pattern was found for reseafch. Those who saw research as 
highly influential in the evaluation and reward system yfere^/ffl&ye likely to 

,be above t^ median in time spent on »reseaich, compared to those^ v/ho saw 
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research as less influential;, the respective proportions are 63 percent 
and 40 percent* again, ^ there appears to be an interaction between 

professional- responsibility, internalized standards for effort, and a 
tendency for faculty to be guided by their perception of relative reward. 

Implications for Changing the Evaluation System * ' - 

We have argued that the University is a professional bureaucracy 
whose^ evaluation process is dominated by profes^sional colleagues. How^^ 
then, could the ^Uhiverfeity- have an » evaluation system* that differed mark^ 
edly from the &ne desired b^ its faculty? *One might assume that the 
facility and the central administration wer^ in oppositipn. with respect 
to producing a more balanced evaluation process, but that simply. was not 
true. The public statements and privajte efforts of the administrators of 
Stanford were, and are, directed towdrd increasing the influence of 
evaluations of teaching* Thus, it appears that everyone wanted change, 
but waited, for some other agent to bring it about* 

The visj.bilit.y of research in a- national and international market 

^ and the lack of visibility of teaching to persons* outside the universiry 
were, probably ^'inaj or factors in preserving the existing reward system^. 

" Since Stanford faculty will continue to participate in a broader social^ 
system ^than the University, and since the. University's boundaries for 
the eA5aluati6n of research are permeable, the faculty and the administra- 
tion must take th^ise realities into account 'in any future attempfts at 
change* But faculty dissatisfaction with the relative de-emphasis of 
teaching TSf-fehje^-evaluat ion system provided an opportunity to rfedress the 

• imbalance between effort and reward* ^ 

f • ' ' ' 

Even if the administration of the University presses for change^ the 

change 'itself must be carried^ out by the fat^ulty — the faculty has the 

power to bring ab^Ut a change or thwart it. The evaluation-reward process 



appears to bp t^^primary means by whicht leadership groups in the admini- 
stration ^d^ the faculty cah i'nitiate change when the change is in .a 
direction approved/by a majority of the faculty. If faculty effbrfe on 
^ny task is desirable, that task should receiW appropriate emphasis in 
the evaluation-reward system* 



^ Under the*. auspices of the Dean of Undergraduate Studies, a new of- 
^ ficjal charged with improving some- aspects of faculty teaching, we will 
survey the atanford faculty to examine the effecl^of recent attempts: to 
change the evaluation^ system increasing the weighty given to teaching. 
The study reported here provides the baseline for the measurement of ^ 



change . 
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